Monte Carlo MCMC: Efficient Inference by Sampling Factors
نویسندگان
چکیده
Conditional random fields and other graphical models have achieved state of the art results in a variety of NLP and IE tasks including coreference and relation extraction. Increasingly, practitioners are using models with more complex structure—higher tree-width, larger fanout, more features, and more data—rendering even approximate inference methods such as MCMC inefficient. In this paper we propose an alternative MCMC sampling scheme in which transition probabilities are approximated by sampling from the set of relevant factors. We demonstrate that our method converges more quickly than a traditional MCMC sampler for both marginal and MAP inference. In an author coreference task with over 5 million mentions, we achieve a 13 times speedup over regular MCMC inference.
منابع مشابه
On Bayesian inference for the M/G/1 queue with efficient MCMC sampling
We introduce an efficient MCMC sampling scheme to perform Bayesian inference in the M/G/1 queueing model given only observations of interdeparture times. Our MCMC scheme uses a combination of Gibbs sampling and simple Metropolis updates together with three novel “shift” and “scale” updates. We show that our novel updates improve the speed of sampling considerably, by factors of about 60 to abou...
متن کاملMonte Carlo MCMC: Efficient Inference by Approximate Sampling
Conditional random fields and other graphical models have achieved state of the art results in a variety of tasks such as coreference, relation extraction, data integration, and parsing. Increasingly, practitioners are using models with more complex structure—higher treewidth, larger fan-out, more features, and more data—rendering even approximate inference methods such as MCMC inefficient. In ...
متن کاملOf Bugs and Birds: Markov Chain Monte Carlo for Hierarchical Modeling in Wildlife Research
Markov chain Monte Carlo (MCMC) is a statistical innovation methodology that allows researchers to fit far more complex models to data than is feasible using conventional methods. Despite its widespread use in a variety of scientific fields, MCMC appears to be underutilized in wildlife applications. This may be due to a misconception that MCMC requires the adoption of a subjective Bayesian anal...
متن کاملMethods of Data Analysis Metropolis Monte Carlo and Entropic Sampling
Many problems in statistical physics, machine learning and statistical inference require us to draw samples from (potentially very) high-dimensional distributions, P (~x). Often, one does not have an explicit expression for the probability distribution but (as we will see) can evaluate a function f(~x) ∝ P (~x). Markov Chain Monte Carlo is a way of sequentially generating samples (in a “chain”)...
متن کاملMCMC and Naive Parallel Gibbs Sampling
In this scribe, we are going to review the Parallel Monte Carlo Markov Chain (MCMC) method. First, we will recap of MCMC methods, particularly the Metropolis-Hasting and Gibbs Sampling algorithms. Then we will show the drawbacks of these classical MCMC methods as well as the Naive Parallel Gibbs Sampling approach. Finally, we will come up with the Sequential Monte Carlo and Parallel Inference f...
متن کامل